Methodological rigor and quality of reporting of clinical trials published with physical activity interventions: A report from the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative)

Background This study addresses the need for improved transparency and reproducibility in randomized clinical trials (RCTs) within the field of physical activity (PA) interventions. Despite efforts to promote these practices, there is limited evidence on the adherence to established reporting and methodological standards in published RCTs. The research, part of the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative) in 2020, assessed the methodological standards and reporting quality of RCTs focusing on PA interventions. Methods RCTs of PA advice or exercise interventions published in 2020 were selected. Monthly searches were conducted on PubMed/MEDLINE targeting six top-tier exercise science journals. Assessments were conducted by two independent authors, based on 44 items originally from CONSORT and TIDieR reporting guidelines. These items were divided into seven domains: transparency, completeness, participants, intervention, rigor methodology, outcomes and critical analysis. Descriptive analysis was performed using absolute and relative frequencies, and exploratory analysis was done by comparing proportions using the χ2 test (α = 0.05). Results Out of 1,766 RCTs evaluated for eligibility, 53 were included. The median adherence to recommended items across the studies was 30 (18–44) items in individual assessments. Notably, items demonstrating full adherence were related to intervention description, justification, outcome measurement, effect sizes, and statistical analysis. Conversely, the least reported item pertained to mentioning unplanned modifications during trials, appearing in only 11.3% of studies. Among the 53 RCTs, 67.9% reported having a registration, and these registered studies showed higher adherence to assessed items compared to non-registered ones. Conclusions In summary, while critical analysis aspects were more comprehensively described, aspects associated with transparency, such as protocol registrations/modifications and intervention descriptions, were reported suboptimally. The findings underscore the importance of promoting resources related to reporting quality and transparent research practices for investigators and editors in the exercise sciences discipline.


Rationale
3 Describe the rationale for the review in the context of existing knowledge.27 Objectives 4 Provide an explicit statement of the objective(s) or question(s) the review addresses.28

METHODS
Eligibility criteria 5 Specify the inclusion and exclusion criteria for the review and how studies were grouped for the syntheses.29 Information sources 6 Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies.Specify the date when each source was last searched or consulted.
Search strategy 7 Present the full search strategies for all databases, registers and websites, including any filters and limits used.

MS
Selection process 8 Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.
Data collection process 9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.

Location where item is reported
Data items 10a List and define all outcomes for which data were sought.Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.

10b
List and define all other variables for which data were sought (e.g.participant and intervention characteristics, funding sources).Describe any assumptions made about any missing or unclear information.

31
Study risk of bias assessment 11 Specify the methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process.

NA
Effect measures 12 Specify for each outcome the effect measure(s) (e.g.risk ratio, mean difference) used in the synthesis or presentation of results.32

Synthesis methods
13a Describe the processes used to decide which studies were eligible for each synthesis (e.g.tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)).

Additional file 4: Exceptions and operationalization of the answers of some items
Items where we considered more than one answer, the answer "No" or another answer as being "positive" (i.e., recommended practice): 1. Abstract: If any quantitative result is reported, is it accompanied by the respective precision estimate (e.g., standard deviation, confidence intervals, etc)?" (Outcome) The possibilities were: "Yes", "No" or "There is no reporting of quantitative results".
Therefore, the last one was considered "No".

Abstract: Is a P-value reported in the abstract?" (Outcome)
The possibilities were: "No", "Yes, with P value(s) reported at varied levels (BOTH equal/lower AND higher than 0.05)" or "Yes, with P value(s) reported at a level EQUAL or LOWER than 0.05".Therefore, these two last possibilities were combined as "Yes", when the result of this question was presented in binary form (i.e., yes or no).

Abstract: Does the abstract inform the primary outcome (variable of interest)?" (Outcome)
The possibilities were: "No", "Yes" or "Unclear".The last one was used when it was not possible to clearly identify the outcomes.This way, "UNCLEAR" was classified as"NO"

Introduction: Is there a hypothesis stated for the outcome(s) of interest? (Critical appraisal)
The possibilities were: "No", "Yes", "Does not apply", or "Unclear" .When there wasn't a directional hypothesis, but there is explanation for an exploratory approach, the answer, this option was considered "YES".Therefore when it was not possible to clearly identify the outcomes, "UNCLEAR" was classified as"NO".
5. Methods: Is there a detailed description of eligibility criteria for participants?(i.e., information that would allow to replicate the inclusion and exclusion decisions)" (Participants) The possibilities were: "No", "Yes" or "Unclear".When it was not possible to clearly identify the eligibility criteria, "UNCLEAR" was classified as"NO".

Methods: Are primary and secondary outcomes listed?" (Outcomes)
The possibilities were: "No", "Yes", "Partially yes", or "Unclear".In this case we assume that, although incomplete, the topic was approached, so "Partially yes"was classified as"YES".
When it was not possible to clearly identify the eligibility criteria, "UNCLEAR" was classified as"NO".The possibilities were: "No", "Yes" or "Partially yes".In this case we assume that, although incomplete, the topic was approached, so "Partially yes"was classified as"YES" The possibilities were: "No", "Yes" or "Unclear".When it was not possible to clearly identify the eligibility criteria, "UNCLEAR" was classified as"NO".The possibilities were: "No", "Yes" or "Partially yes".In this case we assume that, although incomplete, the topic was approached, so "Partially yes"was classified as"YES" 10.Methods/Results-TID9: If any form of individual tailoring for the intervention was used (such as, variable exercise intensity), is there a description for the rationale and guide for tailoring?"(Intervention).
The possibilities were: "No", "Yes", "Does not apply", or "Unclear" ..When there was no individual directional intervention, but there is an explanation for this, the answer, this option was considered "YES".Therefore when it was not possible to clearly identify the outcomes, "UNCLEAR" was classified as"NO".The possibilities were: "No", "Yes" or "Does not apply"..When there was no individual directional intervention, but there is an explanation for this, the answer, this option was considered "YES".
12. Discussion: Is there a potential spin bias based on a specific reporting strategy to highlight that the experimental treatment is beneficial?" The possibilities were: "No" or "Yes".The assessment is NO that there was no spin bias, however, to facilitate the analysis "No" was considered "Yes".

7 .
Methods: Is the type of randomization sufficiently described?(Note: (i) type, such as computer; (ii) allocation ratio; and (iii) methods of restriction (if any) such as stratification or blocking should be considered)" (Methodological rigor)

8 .
Methods-TID3: Is there a description of physical and information materials used as part of the intervention?(i.e., information that would allow to replicate the intervention with materials for participants or intervention providers)" (Intervention) Methods-TID4: Is there a description of activities or procedures used to carry out the intervention?(i.e., information that would allow to replicate the necessary steps to run the intervention)" (Intervention)

9 .
Methods-TID5: Is there a description of individuals involved in providing the intervention?(i.e., information that would allow to replicate the necessary workforce to run the intervention)" (Intervention) Methods-TID6: Is there a description of modes of delivery of the intervention?(i.e., information that would allow to replicate the delivery to run the intervention, such as face-to-face, telefone, individually or in a group)" (Intervention) Methods/Results-TID8: Is there a description regarding the period, amount, intensity and schedule of delivery?(i.e., information that would allow to replicate 'when' and 'how much')" (Intervention)

11 .
Methods/Results-TID11: Is there a description of materials/strategies used in regard to intervention adherence?"(Intervention)[69] Methods/Results/Discussion: Is there any mention of changes to the ORIGINAL protocol during the course of the study?(Note: regards to ANY

Full assessment of RCT with individual items and associated component labels.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al.The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.BMJ 2021;372:n71.Full assessment of RCTs with individual items, associated component labels (i.e., domains) and, whether applicable, the guideline or tool of reference.The items used to evaluate the RCTs were based on (1) what is expected in studies using physical activity interventions or structured exercise programs and (2) presentation of methodological standards for reproducibility purposes, both in clinical practice and in the scientific field.Items selected for assessment of RCTs.
2913b Describe any methods required to prepare the data for presentation or synthesis, such as handling of missing summary statistics, MS Risk of bias in studies 18 Present assessments of risk of bias for each included study.NAResults of individual studies19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g.confidence/credible interval), ideally using structured tables or plots.35Results of syntheses 20aFor each synthesis, briefly summarise the characteristics and risk of bias among contributing studies.NA 20b Present results of all statistical syntheses conducted.If meta-analysis was done, present for each the summary estimate and its precision (e.g.confidence/credible interval) and measures of statistical heterogeneity.If comparing groups, describe the direction of the effect.NA 20cPresent results of all investigations of possible causes of heterogeneity among study results.NA 20d Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results.41Reportingbiases21Presentassessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed.24aProvideregistrationinformationfor the review, including register name and registration number, or state that the review was not registered.NA 24b Indicate where the review protocol can be accessed, or state that a protocol was not prepared.1724cDescribeand explain any amendments to information provided at registration or in the protocol.NA Support 25 Describe sources of financial or non-financial support for the review, and the role of the funders or sponsors in the review.27Reportwhich of the following are publicly available and where they can be found: template data collection forms; data extracted from included studies; data used for all analyses; analytic code; any other materials used in the review.48From: